Phone calls generate large amounts of information. The discussion of work-related topics, schedule of time and place of a meeting, annotation of shopping/to-do lists and contact details of a colleague, among others, often involve meaningful pieces of information which are hard to keep in mind after the conversation. Recalling information exchanged in phone conversations is not an easy task and it usually requires some method for memory aid. Traditionally, people would take hand notes in a paper while they were having a phone conversation, but as people use mobile telephones more and more often, it started to appear automatic methods for recording and summarizing phone calls. Personal notes primarily serve as a memory aid for individuals to remember important facts, actions, ideas, and decisions and the annotations may be as simple as highlighting passages or producing a new written product. It is often the case one has to take notes either during—online—or after—offline—cooperative situations, such as in meetings, lectures or phone calls.
Full recordings of phone calls were once considered to address this problem. Nevertheless, it has been demonstrated that it is not an efficient approach since browsing audio data is a lengthy activity that most people prefer to avoid.
Another later solution, avoiding recording full conversations, focuses on the fact that people often choose annotation tools for faster retrieval, even if that implies lower accuracy in the information recall process. Examples of these tools include audio recorders, mobile phone annotation applications, and the usual pen and paper approach. However, it is often the case when these resources are not available during a phone call, or even that either parties have both hands busy, thus preventing the note-taking activity. Furthermore, taking notes during phone calls implies an additional cognitive load that might lead to loss of conversation threads and break the natural information flow.
There are some services that try to address these issues by allowing users to perform certain actions during the call in order to create live annotations or set markers in the recorded calls. However, these solutions have some inconveniences; such as interrupting the natural information flow of a phone conversation, requiring users to perform specific actions during the call, or simply creating a blob of audio information with more data than it is actually needed.
In the recent years, methods for annotation in work-related meetings have been studied. Notes can be used as memory cues for participants to recall events of a meeting rather than being full recordings of the activity but both attention and active participation is required, and taking notes at the same time may become an additional cognitive load that reduces the person's ability to participate.
These needs have been lately supported by different solutions, including electronic annotation tools that leverage desktop computers or mobile devices, as well as the common paper and pen approach, still frequently used in the form of post-it notes, miscellaneous text files, or the corner of other printed documents.
A phone call could be considered as two-people meeting. Nevertheless, these activities are different from work-related meetings. For example, typical phone calls tend to be relatively shorter, they are frequently not planned beforehand and they lack the structure of a meeting, being plenty instead of a series of salutations and informal dialogs. Also, it has been observed that during phone calls, participants often have their hands busy, either by performing another activity (e.g. driving) or by holding the phone, documents or other objects. Despite these differences, specific annotations for phone calls have received little coverage in the prior art.
Finally, some related research efforts had been done in the process of automatic summarization of texts. Summarization aims to include all the concepts included in a body of text, while reducing the actual amount of text. Summarization then is not selective about the pieces of information that should be included in the final result: it must include every piece, but they must occupy less space. Annotation, on the other hand, aims to select very specific pieces from a body of information while ignoring the remaining ones. As example of an existing solution, the U.S. Pat. No. 7,602,892 “Telephony annotation services” provides a simple method for phone call annotation. This method requires users to remember a set of actions that must be taken to trigger the annotation process. It also requires users to interrupt the normal flow of the conversation to perform these actions, so, in consequence, users might lose pieces of information while performing these actions during a call (e.g. in order to setup audio markers or record a live audio note during a phone call). In addition, the nature of the method could significantly reduce precision in annotations (e.g. audio markers created in real-time by users should frequently have an offset time related to the actual important part of the call). And finally, because of its lack of precision, the resulting recorded information is excessive and would require further editing to obtain the actual annotations.
It is therefore, a lack in the prior art of a method or system to automatically annotates pieces related to phone conversations requiring very few—or none—user interaction at the moment of the call. A lack of a method able to automatically identify and annotate important pieces of information—in the long term—that tend not to be considered relevant in first instance whenever they are not part of the original objective of the call. Somehow, it is needed to organize automatically taken annotations avoiding requiring an immediate user interaction, but later the user should has the chance to decide about said organization by giving his approval or reassigning the relevance of the annotations automatically taken.